Variational Bayesian Inference Techniques

Variational Bayesian Inference Note

When we use EM that uses maximum likelihood as a criterion to select the number of Gaussians, we face the problem of that as the complexity of model increases, the training likihood strictly improves, which means the larger number of Gaussians, the better fit of the training data (see Figure 1). We can see from this example, the traning log-likelihood can even become positive when some clusters...

Variational Inference for Nonparametric Bayesian Quantile Regression

Quantile regression deals with the problem of computing robust estimators when the conditional mean and standard deviation of the predicted function are inadequate to capture its variability. The technique has an extensive list of applications, including health sciences, ecology and finance. In this work we present a nonparametric method of inferring quantiles and derive a novel Variational Bay...

Variational Bayesian Inference with Stochastic Search

Mean-field variational inference is a method for approximate Bayesian posterior inference. It approximates a full posterior distribution with a factorized set of distributions by maximizing a lower bound on the marginal likelihood. This requires the ability to integrate a sum of terms in the log joint likelihood using this factorized distribution. Often not all integrals are in closed form, whi...

Collapsed Variational Bayesian Inference for PCFGs

This paper presents a collapsed variational Bayesian inference algorithm for PCFGs that has the advantages of two dominant Bayesian training algorithms for PCFGs, namely variational Bayesian inference and Markov chain Monte Carlo. In three kinds of experiments, we illustrate that our algorithm achieves close performance to the Hastings sampling algorithm while using an order of magnitude less t...

Variational Bayesian Inference for Financial Models